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DESCRIPTION 

Healing aid, training device, game device and sound output device 

5 Technical Field 

The present invention relates to a hearing aid, a hearing training 
device, a game device and other devices that employ Noise-Vocoded 
Speech Sound obtained by subjecting at least one frequency band signal 
of a sound signal to noise. In particular, the present invention relates 
10 to devices suitable to people with a neural disorder and trainers for 
training for recovery from a neural disorder. 

Background Art 

Conventionally, in some hearing aids, frequency characteristics 
15 compensation or limitation of a dynamic range is performed with respect 
to audio input in accordance with the hearing characteristics that are 
left for a person with hearing disorder in order to improve the 
intelligibility. For people with a neural disorder, such a hearing aid 
may not be capable of providing sufficient intelligibility. 
20 On the other hand, in research on recognition of sound signals, it 

has come to be known that a speech can be recognized to a considerable 
extent without hearing a complete sound signal, that is, even if a 
component of a sound signal is subjected to noise by a predetermined 
method. 

25 According to such a research, a sound signal is divided into four 

frequency bands (0 to 600, 600 to 1500, 1500 to 2500, 2500 to 4000 Hz), 
each sound signal is subjected to half-wave rectification and is applied to 



1 



a 16 Hz lowpass filter so as to obtain an amplitude envelope of each 
frequency band, and is multiplied by a band noise corresponding to each 
frequency band, and the thus obtained signals are added to generate a 
signal. Such a signal is referred to as "Noise Vocoded Speech Sound". 
5 It has been reported that when people with normal hearing ability heard 
the Noise-Vocoded Speech Sound, an intelligibility of about 80% was 
obtained. 

However, conventional hearing aids often do not provide an 
adequate intelligibility to people with neural disorders or some of the 

10 people with hearing disorders. It is necessary to provide a hearing aid 
that has an effect on such people and further allows brain activity to be 
activated. Furthermore, it is necessary to provide a device for training 
the hearing ability of people with a neural disorder or some of the people 
with hearing disorders. Moreover, it is necessary to train people with 

15 normal hearing ability to recognize speeches produced by people with a 
neural disorder. 

i 

Disclosure of Invention 

The present invention provides a hearing aid in which at least 

20 one portion of an input sound signal is divided into frequency band 
signals and a single or a plurality of the frequency band signals are 
subjected to noise to generate a Noise-Vocoded Speech Sound signal, 
which a user can hear. Such a hearing aid facilitates activation of the 
brain and is expected to provide an effect on treatment or training of 

25 people with a neural disorder. Such a hearing aid lets a Noise-Vocoded 
Speech Sound signal recognized by utilizing normal portions of the brain 
to the maximum level and the Noise-Vocoded Speech Sound signal is 



2 



compensated for by other normal portions of the brain, in order to let a 
person with hearing difficulty understand the meaning of the input 
sound. 

The present invention provides a training device in which a 
5 Noise-Vocoded Speech Sound signal obtained by dividing at least one 
portion of a sound signal into frequency band signals and subjecting a 
single or a plurality of the frequency band signals to noise to a trainee, 
the trainee pronounces the recognized words and learns the correctness. 
Such a training device has an effect of promoting the activation of the 

10 brain activity. Such a training device is useful in training a trainee to 
understand language and, for trainers, this is useful in training for 
improving the training ability. Such a hearing aid lets a Noise-Vocoded 
Speech Sound signed recognized by utilizing normal portions of the brain 
to the maximum level and the Noise-Vocoded Speech Sound signal is 

15 compensated for by other normal portions of the brain, in order to let a 
person with hearing difficulty understand the meaning of the input 
sound. 

The present invention provides a game device in which a 
Noise-Vocoded Speech Sound signal obtained by dividing at least one 

20 portion of a sound signal into frequency band signals and subjecting a 
single or a plurality of the frequency band signals to noise to a game 
player, the game player pronounces the recognized words and competes 
in the ratio of correctness or the number of correct answers. Such a 
game device has an effect of promoting the activation of the brain 

25 activity. Such a training device is useful in training a trainee to 
understand language and, for trainers, this is useful in training for 
improving the training ability. Apart from training, such a game device 
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can be a game device for normal people as a game for guessing words or 
sentences from the Noise-Vocoded Speech Sound. 

Furthermore, a sound output device of the present invention 
generates a Noise-Vocoded Speech Sound signal in which a component of 
5 a sound source signal is subjected to noise by extracting a signal with a 
predetermined frequency band from the sound source signal by a first 
band filtering portion having a plurality of band filters," extracting an 
amplitude envelope of each frequency band signal by an envelope 
extracting portion having an envelope extractor; applying a noise source 

10 signal to a second filtering portion having a plurality of band filters to 
extract a noise signal corresponding to the predetermined frequency 
band; multiplying an output from the first band filtering portion by an 
output from the second band filtering portion in a multiplying portion; 
and accumulating outputs from the multiplying portion in an adding 

15 portion. If this sound output device is configured so that the number of 
band filters or the frequency boundary of the frequency bands can be 
selected or changed, this sound output device can be used for various 
purposes. Furthermore, with automatic language recognition, when the 
number of band filters or the frequency boundary of the frequency bands 

20 is selected or changed suitably with a language, this device can be used 
by a plurality of peoples with different nationalities, and can be used for 
training of foreign languages. 

The hearing aid, the training device, and the game device of the 
present invention can be implemented with the procedures of a computer 

25 program or the like, and therefore can be made into a program recording 
medium storing a program that is executed by a computer or a program 
that is executed by a computer. 
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Brief Description of Drawings 

FIG. 1 is a block diagram of a hearing aid according to one 
embodiment of the present invention. 
5 FIG. 2 is a block diagram of a training device according to one 

i 

embodiment of the present invention. 

FIG. 3 is a block diagram of a training device or 
a game device according to one embodiment of the present invention. 

FIG. 4 is a flowchart showing the behavior of a hearing aid 
10 according to one embodiment of the present invention. 

FIG. 5 is a flowchart showing the behavior of a training device or 
a game device according to one embodiment of the present invention. 

FIG. 6 is a block diagram of a hearing aid according to one 
embodiment of the present invention. 
15 FIG. 7 is a block diagram of a training device according to one 

embodiment of the present invention. 



Best Mode for Carrying Out the Invention 

Hereinafter, embodiments of a hearing aid and the like will be 
20 described with reference to the accompanying drawings. It should be 
noted that components bearing the same reference numeral in the 
embodiments perform the same operation and may not be described in 
duplicate. 



25 Embodiment 1 

FIG. 1 is a block diagram of a hearing aid of the present 
invention. In FIG. 1, a sound signal input from a microphone is applied 



to a band filtering portion 1 via an input terminal 7. The band filtering 
portion 1 has a plurality of band filters la, lb, lc and Id for extracting 
signals having a predetermined frequency band. Output signals from 
the band filters la, lb, lc and Id are applied to envelope extractors 2a, 
5 2b, 2c and 2d, respectively, of an envelope extracting portion 2 so that 
the amplitude envelope of each frequency band signal is extracted. A 
noise signal output from a noise source 5 is applied to a band filtering 
portion 4 having a plurality of band filters 4a, 4b, 4c and 4d, to be 
divided into noise signals having the same frequency bands as those of 

10 the band filtering portion 1. The outputs from the envelope extractors 
2a, 2b, 2c and 2d and the outputs from the band filters 4a, 4b, 4c and 4d 
are multiplied for each corresponding band in a multiplying portion 3 
having multipliers 3a, 3b, 3c and 3d. The multiplied results are 
summed up in an adding portion 6, and then become output signals in an 

15 output terminal 8. 

When the thus-created NoiseVocoded Speech Sound is emitted 
through an earphone for listening, some of the people with hearing 
difficulty (e.g., hearing difficulty with a neural disorder) may experience 
a better intelligibility than with conventional hearing aids, so that the 

20 Noise-Vocoded Speech Sound can be used for a hearing aid. More 
specifically, this allows people with a neural disorder to recognize the 
Noise-Vocoded Speech Sound with normal portions of the brain, and the 
brain (in general, portions different from the normal portions) to 
compensate the portion of audio information that can be recognized so 

25 that the input sound can be understood. In other words, according to 
the above-described hearing aid, the hearing ability can be recovered by 
utilizing the normal portions of the brain effectively to the maximum. 
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Furthermore, such a Noise-Vocoded Speech Sound is expected to 
activate the activity of the brain of a person with a neural disorder, and 
not only a function of compensation of hearing ability, but also an effect 
of training for recovery of a brain with disorder or treatment thereof can 
5 be expected. Furthermore, in the case of a person with a hearing 
difficulty who does not have a neural disorder but has a disorder in the 
acoustic conduction system of the external ear or the middle ear or in the 
middle from the acoustic perception system of the internal ear to the 
central nervous system, the Noise-Vocoded Speech Sound can allow the 

10 brain to enhance the function of compensating the disorder in the middle 
so as to improve the illegibility. 

Next, a variation of the hearing aid will be described. For 
example, when a person with hearing difficulty has a poor sensitivity to 
a certain frequency band, the frequency characteristics that compensate 

15 for a reduction in the sensitivity to that frequency can be provided to the 
frequency characteristics of each band filter. Furthermore, in the case 
where there is a limit on the range of the sound volume that can be 
heard so that sound cannot be heard in a small sound volume, and sound 
cannot be recognized because it is distorted in a large sound volume, 

20 then the non-linear multiplication characteristics for correcting the 
dynamic range as appropriate can be provided to each of the multipliers 
3a, 3b, 3c and 3d of the multiplying portion 3. 

In FIG. 1, the sound signals of all of the four frequency bands are 
replaced by frequency band noise signals, but sound signals of a part of 

25 the frequency bands are not supplied to the multiplying portion 3 but 
supplied directly to the adder 6, so that sound signal components can be 
left. Also with respect to each of the frequency bands whose sound 
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signal components are to be left, a correction corresponding to 
degradation of the dynamic range or the frequency characteristics of 
hearing disorder may be performed. 

As described above, according to the present invention, a hearing 
5 aid can be provided in which at least one portion of an input sound 
signal is divided to a single or a plurality of frequency band signals and 
subjected to noise to generate Noise-Vocoded Speech Sound signals, 
which a user can hear, and the activation of the brain can be facilitated 
by this hearing aid so that an effect on treatment and training of people 
10 with a neural disorder can be expected. 

Embodiment 2 

FIG. 2 is a block diagram of a training device using the 
Noise-Vocoded Speech Sound of the present invention. In FIG. 2, the 

15 band filtering portion 1, the envelope extracting portion 2, the 
multiplying portion 3, and the band filtering portion 4, the noise source 5, 
and the adding portion 6 have the same configurations as in those FIG. 1. 
The sound source signal portion 10 stores sound signals of various words 
or sentences. A sound source selection control portion 11 selects and 

20 designates a sound signal of a predetermined word or sentence by 
supplying a control signal to the sound signal portion 10. The sound 
source signal portion 10 outputs the sound signal of the designated word 
or sentence to the band filtering portion 1. The Noise-Vocoded Speech 
Sound signal of that word or sentence is obtained from an output 

25 terminal 8. This is listened to by a trainee. A trainer operates the 
sound source selection control portion 11 according to a training program, 
and has a trainee listen to the words or the sentences sequentially, and 
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the trainee listens to the Noise Vocoded Speech Sounds and pronounces 
the recognized words or sentences to the trainer. The trainer judges 
whether they are correct or incorrect and lets the trainee know the 
results, and then proceeds to the next step. The trainee learns by being 
5 informed of whether their responses are correct or incorrect. The 
trainer selects words or sentences that are to be supplied next, 
depending on their accuracy. 

For the frequency bands of the band filters of the band filtering 
portions 1 and 4, 0 to 600 Hz, 600 to 1500 Hz, 1500 to 2500 Hz, and 2500 

10 to 4000 Hz are used as the standard frequency bands. A band selecting 
portion 12 can switch the above-described frequency bands of the band 
filtering portions 1 and 4. For example, the number of frequency bands 
can be selected among 1, 2, 3, and 4. This is because the four frequency 
bands are necessary, depending on the words, that is, whether the sound 

15 is a vowel sound, a consonant, an explosive sound or the like. For 
example, the number of the frequency bands can be 2 by setting the 
output from the band filters lc, Id, 4c and 4d to 0. Furthermore, with 
respect to all or any part of the band filters, the boundary frequency 
defining the frequency bands can be switched to other frequencies than 

20 600 Hz, 1500 Hz, 2500 Hz, and 4000 Hz. 600 Hz, 1500 Hz, 2500 Hz, and 
4000 Hz are close to the standard frequency boundary that separates the 
vowel sounds of sound, that is, /a/, /i/, /u/, /e/, and lol in Japanese by the 
first and the second formants. However, the frequency boundary may 
be slightly different from person to person. It is expected to improve the 

25 training effect by adjusting and changing the boundary of the frequency 
bands in accordance with such a difference between, individuals, so that 

* ■ m 

the device is configured such that the boundary can be switched and 
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selected. Furthermore, foreign languages have a vowel system different 
from that of Japanese, so that the device may be configured such that 
the number of the band filters or the boundary frequency can be 
switched in order to be used with foreign languages. 
5 For use with foreign languages, a language automatic recognition 

portion may be provided and automatically recognize words or sentences 
that have been input first by a trainee and a trainer through a 
microphone. Then, the language automatic recognition portion may 
supply data about the country to the band selecting portion 12, and the 

10 band selecting portion 12 may set the number of the band filters and the 
frequency band boundary corresponding to the language of the country in 
the band filtering portion 1 and 4. 

It should be noted that selection or switching of the number of 
the band filters and the frequency band boundary^ and selection or 

15 switching of the number of the band filters and the frequency band 
boundary by the language automatic recognition portion as described 
above can be applied to the hearing aid described in Embodiment 1. 

As described above, the present invention provides an effect of 
facilitating the activation of the activity of the brain by the 

20 above-described training device. The present invention is useful to 
train a trainee to understand language or to train a trainer to improve 
the training ability. 

Embodiment 3 

25 FIG. 3 is a block diagram of a training device using the 

Noise-Vocoded Speech Sound of the present invention. Referring to this 
block diagram, the aspects that are different from FIG. 2 will be 
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described. The Noise-Vocoded Speech Sound output from the adding 
portion 6 is presented to a trainee through a headphone 13. The sound 
source selection control portion 11 has a function of controlling 
generation of display signals, in addition to the function of selecting a 
5 sound source signal, and displays an instruction to a trainee or a 
response to words or sentences with Noise-Vocoded Speech Sound with 
texts on a screen of a display device 14. A response input portion 15 is a 
keyboard with which recognized words or sentences are input. 
Information signals from the response input portion 15 are transmitted 

10 to the sound source selection control portion 11, and the sound source 
selection control portion 11 analyses the content of the response and 
selects words or sentences that are to be presented next, according to the 
response results. 

A training program is contained in the sound source selection 

15 control portion 11. For example, 10 basic words or sentences constitute 
one set, and words or sentences are presented sequentially one by one. 
A trainee listens to the Noise-Vocoded Speech Sound thereof and inputs 
recognized words or sentences as a response with Japanese characters to 
the response input portion 15. The sound source selection control 

20 portion 11 counts correct and incorrect responses and displays correct 

* 

answers on the display device 14 and presents again the Noise-Vocoded 
Speech Sound at the same time. After the presentation of and the 
responses to the ten words or sentences, the ratio of the correct 
responses is displayed. Finally, the ten words or the sentences of the 

4 

25 problems are presented again for confirmation. Thus, the trainee can 
learn by himself without a trainer. The sound source selection control 
portion 11 starts a next problem program having an appropriate 
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difficulty, depending on the ratio of the correct responses. 

When presenting the Noise-Vocoded Speech Sound, its sound may 
be displayed Ln the display device with texts. For example, a correct 
sentence and a plurality of partially incorrect sentences that may be 
misheard easily may be displayed, and a trainee may input the number 
assigned to the sentence that he/she thinks correct to the response input 
portion 15. 

The selection of the filters in the band filtering portions 1 and 4, 
the selection and switching of the boundary of the band frequencies and 
the selection and switching by the language automatic recognition 
portion, which are described in Embodiment 2, can be applied to this 
embodiment. 

According to this embodiment, with the above-described training 
device, trainees can be trained by themselves. 

Embodiment 4 

The learning procedure with the configuration of FIG. 3 is also a 
kind of game. A game device can be realized with the above-described 
configuration as the basis. First, the title of a game or a menu for 
selecting the degree of difficulty are displayed, and a game player selects 
the degree of difficulty from the response input portion 15, and the sound 
source selection control portion 11 selects the Noise-Vocoded Speech 
Sounds of words or sentences with the selected degree of difficulty. The 
ratio of correct responses may be recorded or the number of correct 
responses may be displayed in the display device 14 during a game. 
When a high mark is obtained, an amusing view appears on the screen 
as a prize so that the play can be amused. It can be designed in the 
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form of a competition as to how many questions were answered in a 
predetermined period of time or how many correct responses were 
provided. In such a game, a game device may be designed for 
handicapped people or may be designed for normal people. The 
presented content, the presentation speed, and the presented view can be 
designed as appropriate, depending on the person who uses it. 

The game player may select the degree of difficulty of the game 
by operating the band selecting portion 12. For example, the number of 
filters may be selected among 1 to 4. The original words or sentences 
can be understood more easily in the case of four band filters than in the 
case of one band filter, so that the game player can select the level of the 
difficulty of the game. 

Embodiment 5 

» . 

Hereinafter, an embodiment of a method for hearing with an aid 
will be described. FIG. 4 is an example of a flowchart of a method for 
implementing a function of the hearing aid according to the present 
invention. 

In the sound input procedure (SlO), an input sound signal from a 
microphone of the hearing aid is ADxonverted for preparation to supply 
sound data to the band filtering procedure (Sl2). This procedure is 
steadily performed hereinafter. Next, in the band selection procedure 
(S 11), the number of the band frequencies or the boundary frequencies of 
the band frequencies for band filtering that is performed in the band 
filtering procedures (S12) and (Sl4) below is changed or set, if necessary. 
This procedure is performed in response to an operation by a user. If 
there is such an operation, this procedure is omitted. Next, in the band 
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filtering procedure (S12), sound data is filtered, based on the number of 
the band frequencies or the boundary frequency of the band frequencies 
of the band filtering that are set. In the envelope extraction procedure 
(Si 3), the envelope component data of the filtered sound data is 
extracted. Then, in the band filtering procedures (S14), noise signals of 
white noise are filtered, according to the number of the band frequencies 
or the boundary frequencies of the band frequencies for band filtering 
that are set, so that band noise signal data is generated. The envelope 
component data and the band noise signal data are multiplied in the 
following multiplication procedure (Sl5), and when the multiplication 
results are obtained for a plurality of frequency bands, they are 
accumulated in the adding procedure (SlG). The accumulated sound 
data is Noise-Vocoded Speech Sound data. In the signal presentation 
procedure (S17), this is presented to the user through an earphone in the 
form of a DA-converted analog sound signal for listening. 

The procedures (SlO) to (Si 7) may be performed sequentially 
according to FIG. 4, or may be performed in parallel. These procedures 
can be realized in the form of a program of a digital signal processor 
CDSP). 

The procedures (Sll) to CS16) constitute a Noise-Vocoded Speech 
Sound generation procedure (S100). 

The band selection procedure (Sll) may be provided with a 
language automatic recognition procedure for, for example, Japanese, 
English, German or Chinese, and the number of the frequency bands and 
the boundary frequencies defining the frequency bands that correspond 
to the recognized language may be selected. The technique for language 
automatic recognition is known and therefore is not described in detail 
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here. 

Embodiment 6 

Hereinafter, an embodiment of a training method will be 
5 described. FIG. 5 is an example of a flowchart of the procedures that 
realize the functions of the training device of the present invention. 

When a training program is started, in the sound source selection 
procedure (S20), a predetermined word or sentence that is to be 
presented is selected from the sound source signal data and supplied to 

10 the Noise-Vocoded Speech Sound generation procedure (S100). The 
Noise Vocoded Speech Sound generation procedure (S100) generates 
Noise-Vocoded Speech Sound data. In the Noise-Vocoded Speech Sound 
presentation procedure (S2l), the generated Noise-Vocoded Speech 
Sound is converted to an analog sound signal, which is listened to by a 

15 trainee through an earphone. In the response procedure (S22), the 
trainee inputs a word or a sentence that is understood by recognizing the 
Noise-Vocoded Speech Sound that the trainee heard to the response 
input portion 15. An evaluation device determines whether or not the 
response data that has been input is correct in a response evaluation 

20 procedure (S23), and in a correct answer presentation procedure (S24), 
the result of correct or in correct or a correct word or sentence is 
displayed in the display device. At this point, training for one question 
ends. A training program menu constituted by a series of questions can 
be implemented by repeating these procedures in this order. 

25 

Embodiment 7 

Hereinafter, an embodiment of a game method will be described. 
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If the ratio of correct responses or the scores acquired from correct 
responses is displayed in the correct answer presentation procedure 
(S24) of the training device procedure of FIG. 5, procedures for a game 
method can be obtained. 

Embodiment 8 

In the hearing aid of FIG. 1, an input signal from a microphone is 
applied to the band filtering portion 1 through an input terminal 7. 
However, the input signal may contain an ambient noise component, in 
addition to a sound component. In this case, the configuration of FIG. 6 
can be used. In FIG. 6, the input signal applied to the input terminal 7 
from the microphone is applied to the band filtering portion 1 through a 
sound signal extracting portion 9. The sound signal extracting portion 9 
has the function of extracting sound signals from input signals 
containing ambient noise. Therefore, for example, a technique such as 
spectrum subtraction is used to suppress noise components other than 
sound signals contained in input signals. 

Embodiment 9 

In the training device of FIG. 2 or the training device of FIG. 3 
and the game device, when the signals of the sound source signal portion 
10 contain ambient noise other than sound components, the signals may 
be applied to the band filtering portion 1 through the sound signal 
extracting portion 9 that is described with reference to FIG. 6. 
Furthermore, in a training device or a game device in which one of two 
people inputs a word or a sentence through the microphone, and the 
other person listens to the Noise -Vocoded Speech Sound thereof and 
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guesses the original word or sentence, ambient noise may be mixed, so 
that it is preferable to provide the sound signal extracting portion 9. 

Embodiment 10 

5 FIG. 7 shows a configuration in which signals obtained by 

converting sound signals to Noise-Vocoded Speech Sounds are previously 
stored in the sound source signal portion 10 of the training device of FIG. 
3 or the game device, and a trainee listens to an output signal therefrom 
through a headphone 13. Therefore, it is not necessary to provide the 

10 band filtering portions 1 and 4, the envelope extracting portion 2, the 
multiplying portion 3, the noise source 5, the adding portion 6, the band 
selecting portion 12 in FIG. 3. The same configuration can be used in 
the training device of FIG. 2. 

In all of the above-described embodiments, the number of the 

15 band filters of the band filtering portions 1 and 2 is 4 as a typical 
example. However, the number thereof is not limited to 4 and can be 4 
or less or more. The number of the frequency bands that is suitable, 
depending on the situation, can be used. 

A recording medium on which a program for a procedure for a 

20 hearing aid method, a procedure for a training method, or a procedure 
for a game method of the present invention is recorded includes ROM, 
RAM, flexible disks, CD-ROM, DVD, memory cards, hard disk on which 
the program is recorded. Furthermore, the medium includes 
communication media sxich as telephone networks, and transmission 

25 paths. 
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Industrial Applicability 

The present invention relates to a hearing aid, a hearing training 
device, a game device and other devices that employ Noise-Vocoded 
Speech Sound obtained by subjecting at least one frequency band signal 
5 of a sound signal to noise. In particular, the present invention relates 
to devices suitable to people with a neural disorder or trainers for 
training for recovery from a neural disorder. 
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